Summary

This PR combines the core publisher-proxy streaming optimization with the
Next.js RSC follow-up work.

At the platform level, Trusted Server moves from a fully buffered proxy model
to chunked streaming using Fastly stream_to_client(), enabling early header
flush and incremental HTML delivery to reduce TTFB and improve subresource
discovery.

On top of that foundation, the HTML pipeline now supports RSC-aware lazy
accumulation: non-RSC content continues to stream immediately, while only RSC
content that requires post-processing is buffered and rewritten safely. This
preserves correctness for fragmented/cross-script RSC payloads while restoring
meaningful streaming behavior.

Key Changes

• stream_to_client() Integration (publisher.rs)
  Replaced fully buffered response collection with stream_to_client() to
  enable immediate header dispatch and incremental chunk streaming.

• lol_html Output Pipeline (streaming_processor.rs)
  Refactored the HtmlRewriter adapter to implement the OutputSink trait
  with a shared Rc<RefCell<Vec<u8>>>, enabling true incremental streaming.

• Buffer Pre-allocation
  Replaced std::mem::take with Vec::with_capacity and
  std::mem::replace to reduce reallocation churn during chunk processing.

• WASM Hostcall Batching
  Wrapped StreamingBody output in an 8KB std::io::BufWriter to reduce
  WASM-to-host boundary crossings.

• RSC Lazy Accumulation (html_processor.rs)
  Added conditional accumulation mode that starts buffering only when
  post-processing is required (for example, RSC placeholders or fragmented
  scripts). Non-RSC pages continue streaming instead of being fully buffered.

• RSC Post-processing Triggers (nextjs integration)
  Added needs_accumulation support to integration post-processors and
  needs_post_processing detection in placeholder state, including fragmented
  script tracking for fallback re-parse correctness.

• Memory Safety Guardrail
  Added a 10MB cap for accumulated post-processed HTML to avoid unbounded
  memory growth on large/malicious documents.

• Routing and Header Consistency (fastly/src/main.rs, publisher.rs)
  Centralized route classification and standardized response-header application
  across buffered and streaming paths.

• RSC Fixture/Test Expansion
  Added fixture-driven Next.js integration tests (including real Next.js output)
  plus a dedicated example app and scripts for fixture capture and live streaming
  validation.

• Code Health
  Resolved associated Clippy warnings and added missing # Errors
  documentation in streaming-related handlers.

Test Plan

• Local Unit & Workspace Tests
  Run:

  cargo test --workspace

• TypeScript Bundle Build
  Run:

  npm run build

  in crates/js/lib to verify successful generation of integration modules.

• Next.js RSC Integration Tests
  Run:

  cargo test --test nextjs_integration -- --nocapture

  to validate URL rewriting correctness and streaming behavior across fixture
  sets/chunk sizes.

• Local Fastly Simulation
  Run:

  fastly compute serve

  Verify:

  • Headers are correctly injected on streamed responses
  • Proxy behavior remains correct
  • Baseline TTFB improvements (for example, via curl)

• Staging Load Testing
  Execute:

  ./scripts/benchmark.sh

  against staging to quantify external TTFB and Time-to-Last-Byte (TTLB)
  improvements under concurrent traffic.

  Closes

closes #320